Generating and/or prioritizing pre-call content for rendering when awaiting acceptance of an incoming call

ABSTRACT

Implementations set forth herein relate to generating a pre-call analysis for one or more users that are receiving and/or initializing a call with one or more other users, and/or prioritizing pre-call content according to whether security-related value was gleaned from provisioning certain pre-call content. One or more machine learning models can be employed for determining the pre-call content to be cached and/or presented prior to a user accepting a call from another user. Feedback provided before, during, and/or after the call can be used as a basis from which to prioritize certain content and/or sources of content when generating pre-call content for a subsequent call. Other information, such as contextual data (e.g., calendar entries, available peripheral devices, location, etc.) corresponding to the previous call and/or the subsequent call, can also be used as a basis from which to provide a pre-call analysis.

BACKGROUND

Wireless communications have enabled various users to spend boundlesstime communicating with each other through a variety of differentmodalities. Enabling users to communicate in this way can provideconvenience, but can also expose certain users to security risks due tolimitations of network communications. Oftentimes, a user may receive anincoming call from a number that they have no knowledge upon which toguide their decision to accept the incoming call. Should the incomingcall be from a source associated with criminal activity and/or otherdistressing activity, any resources spent mediating the networkconnection would be wasted, as there may be no utility in participatingin the call. For instance, when such calls occur regularly for apopulation of users, network bandwidth would be consumed byinconsequential calls, thereby causing other more important calls to beincidentally re-directed over the network.

When the user is familiar with a source of an incoming call, the usermay accept the call under the assumption that the call will have someamount of utility to themselves or the source of the call. However,during the call, users will often refer to other sources of information,such as other applications and/or devices, in order to refresh theirrecollection of any subject matter relevant to the call. Suchreferencing of other sources external to the computing device that isactively participating in the phone call can prolong the call and causemultiple overlapping layers of network activity. For instance, a userthat is attempting to access, during a call, information that isrelevant to the call, can cause requests for data over a network that isthe same as, or different from, the network handling the call. This canresult in excess usage of bandwidth during the call and may, where thenetwork is the same, degrade the quality of the call. Moreover, the callcan be prolonged as a result of the user needing to access, during thecall, the information that is relevant to the call.

SUMMARY

Implementations set forth herein relate to generating a pre-callanalysis for one or more users that are receiving and/or initializing acall with one or more other users; and/or prioritizing pre-call contentaccording to whether security-related value was gleaned fromprovisioning certain pre-call content. In some implementations, acontext of a user can be characterized by data, which can be accessed infurtherance of determining pre-call content that should be presented tothe user, should the user receive a call while being subject to thecontext. The data characterizing the context can be systematicallydetermined before, during, and/or after a phone call is received by auser from another user and/or device. For instance, when an activityassociated with both a first user and a second user is performed,contextual data can be generated in response to the activity beingperformed. The activity can be, but is not limited to, the first usercreating a calendar event in which the second user is identified as aninvitee. Subsequent to the activity (e.g., the first user creating thecalendar event) being performed, the second user can initialize a phonecall to the first user, and when the call is awaiting acceptance by thefirst user, the first user can be presented with at least a portion of,or a synopsis of, the contextual data. This can eliminate the need forthe user to prolong their phone call or to use call bandwidth byreferencing the contextual data via some separate modality (e.g., aninterface of another computing device, or another application) duringthe phone call. The method further provides an improved user interfacewhich takes, as input, the signal of the received phone call and usesthe input as a trigger to cause a computing system to fetch relevantdata to provide to the user in advance of the user receiving the phonecall. In addition, this can preserve processor resources, as the usermay no longer need to invoke a separate application for referencingrelevant information during a call, but, rather, can rely on pre-callcontent for refreshing their recollection prior to accepting the call.Additionally, when a user is travelling, for example using public orpersonal transport, while telephone signal may be available over theduration of the route, access to suitable data connections (such as WiFior cellular data connections) may be intermittent. As such, while it maybe possible to take a telephone call, it may not be possible to obtainadditional information at all points during the call. By pre-fetchingpre-call content in advance of the call, there is an increasedlikelihood that information is available for use during the call.

In some implementations, a user interface may provide one or more dialogor menu options to facilitate a user to enable the fetching of pre-callcontent. In some implementations, fetching of pre-call content may beselectively enabled based upon one or more parameters selected by theuser. For example, a menu option may be provided for a user to enablepre-call content to be fetched only during predetermined times of theday, only for certain contacts (or for anyone except certain contacts),when the user is within defined geographic boundaries, in dependenceupon whether a user is moving (e.g. driving or on public transport), independence upon whether the user is currently using one or more otherapplications on the or another computing device, etc.

In some implementations, contextual data can be gleaned from variousdifferent sources (e.g., sources associated with different users) inorder to aggregate data that can be used when prioritizing content forprovisioning to a particular user that is receiving a related incomingcall. For example, when one or more other users have limited theirengagement with a particular call source because of apparent criminaland/or other distressing activity, pre-call provisioning of content thatis based on such contextual data can put the user on notice that thesource of the call has been associated with apparent criminal and/orother distressing activity. In this way, the particular user couldsimilarly limit their engagement with the source of the incoming call.

In some implementations, and with explicit, prior permission from auser, dialog content of a call between the user and one or more otherusers can be a basis for provisioning pre-call content. Such dialogcontent can be used as a basis from which to assign and/or modify apriority of content data (and/or a source of content data) forprovisioning to the user when the user is receiving and/or initializinga call with the one or more other users. As an example, when a firstincoming call is awaiting acceptance from a user, an interface of acomputing device can present the user with first content that isassociated with a source of the incoming call and/or that is predictedto be relevant to the first incoming call. The user can participate inthe first incoming call and dialog content of the first incoming callcan be processed in furtherance of comparing a relevance of the firstcontent to the dialog content. The processing can be undertaken in orderto generate feedback for one or more trained machine learning modelsthat can be used when generating data from which the pre-callprovisioning of content is based. For instance, when the first contentis determined to be relevant to the dialog content, a priority of thefirst content can be prioritized over second content, at least if thesecond content was previously assigned a priority that was higher thanan assigned priority of the first content. Additionally, oralternatively, when the first content is determined to be irrelevant tothe dialog content, the priority of the first content can bedeprioritized relative to a previously assigned priority of the secondcontent. As a result, when the user receives a subsequent incoming callfrom the other user, an assigned priority for each content of the firstcontent and the second content can be used as a basis for renderingpre-call content corresponding to the second content, and/or the firstcontent. In some implementations, the dialog content can optionally bebased on an input from a single participant of the call. The input canbe isolated such that only the single participant of the call, and noother input from other participants, is incorporated into the dialogcontent that is used when generating pre-call content. In someimplementations, the dialog content based on input from a singleparticipant can be transcribed and the transcribed input can be used asa basis from which to generate pre-call content. In someimplementations, dialog content can be based on input from only thoseusers that have previously provided explicit permission to have theirinputs to a call be used as a basis from which to generate pre-callcontent. In other words, when dialog content is based on inputs frommultiple users, pre-call content subsequently generated for those userswill be based on inputs from different users.

In some implementations, relevance of pre-call content to an interactionbetween two or more users can be based on feedback provided by at leasttwo users of the two or more users. The feedback can be, but is notlimited to, gestures and/or other actions performed by a particularuser: prior to the pre-call content being presented, while the pre-callcontent is being presented, during the call, and/or after the call. Forexample, pre-call content for a user can be rendered at a touch displaypanel of a computing device when the computing device, or anothercomputing device, is awaiting acceptance of an incoming call fromanother user. While the pre-call content is rendered for the user, theuser can provide a gesture to the touch display panel, indicating aninterest in a certain portion of the pre-call content. In anotherexample, an indication can be rendered of multiple possible types ofpre-call content that may be retrieved when the computing device, oranother computing device, is awaiting acceptance of an incoming callfrom another user. While the indication of possible pre-call content isrendered, the user can provide a gesture indicating an interest in aparticular one of the types of pre-call content. The selected type ofpre-call content may then be retrieved. Implementations set forth hereinmay facilitate continued and guided human-machine interaction inresponse to receipt of a call request. Furthermore, when the user andthe other user are participating in the call, the other user can providea natural language input to their other computing device, and thenatural language input can have one or more correlations to the pre-callcontent. The pre-call content and dialog content of the call, with priorpermission from the user and the other user, can be compared in order toidentify the one or more correlations. Furthermore, the gesture input bythe user and the natural language input by the other user can beconsidered positive feedback, from which the pre-call content can beassigned a priority and/or other pre-call content can be generated. Forinstance, prior to a subsequent call between the user and the otheruser, one or more learning models stored on the computing device (and/ora separate computing device) can be used to generate further pre-callcontent, and/or prioritize the previous pre-call content, based on thebased on the positive feedback. As a result, when the subsequent call isinitialized by the user and/or the other user, an interface of thecomputing device, and/or another interface of another computing device,can provide pre-call content based on the feedback from both the userand the other user.

In some implementations, pre-call content can be cached in memory of oneor more computing devices and systematically adapted over time based ona variety of different factors. For instance, pre-call content can begenerated for provisioning when a user is receiving an incoming callfrom another user, and the pre-call content can be generated prior tothe incoming call, and adapted over time. The pre-call content can beadapted over time in furtherance increasing a probability that thepre-call content will be relevant to any subsequent call received fromthe other user. As an example, first pre-call content can be generatedbased on application data (e.g., a chat application) that characterizesan interaction between a user and another user. Therefore, should theuser receive an incoming call initialized by the other user, the firstpre-call content can be presented to the user in furtherance ofrefreshing a recollection of the user prior to accepting the incomingcall. However, should the user engage with another application (e.g., acalendar application) in order to generate other data that provides somecorrespondence between the user and the other user, second pre-callcontent can be generated based on other data. Thereafter, should theuser receiving an incoming call initialized by the other user, thesecond pre-call content can be presented to the user in furtherance ofrefreshing the recollection of the user based on the second pre-callcontent over the first pre-call content.

The above description is provided as an overview of some implementationsof the present disclosure. Further description of those implementations,and other implementations, are described in more detail below.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by one or more processors(e.g., central processing unit(s) (CPU(s)), graphics processing unit(s)(GPU(s)), and/or tensor processing unit(s) (TPU(s)) to perform a methodsuch as one or more of the methods described above and/or elsewhereherein. Yet other implementations may include a system of one or morecomputers and/or one or more robots that include one or more processorsoperable to execute stored instructions to perform a method such as oneor more of the methods described above and/or elsewhere herein.

It should be appreciated that all combinations of the foregoing conceptsand additional concepts described in greater detail herein arecontemplated as being part of the subject matter disclosed herein. Forexample, all combinations of claimed subject matter appearing at the endof this disclosure are contemplated as being part of the subject matterdisclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A and FIG. 1B each illustrate a view of a user being presentedpre-call content when the user is receiving an incoming call fromanother user.

FIG. 2A, FIG. 2B, and FIG. 2C illustrate a user changing contexts andcausing pre-call content to be generated based on the changes incontext.

FIG. 3 illustrates a system for providing pre-call content to one ormore users that are expected to receive an incoming call.

FIG. 4 illustrates a method for providing pre-call content to a useraccording to one or more different approaches.

FIG. 5 is a block diagram of an example computer system.

DETAILED DESCRIPTION

FIG. 1A illustrates a view 100 of a user 114 being presented pre-callcontent 102 when the user 114 is receiving an incoming call from anotheruser 122. The other user 122 can initialize the call using a computingdevice 124, which can communicate with a computing device 116 over anetwork 120. When the other user 122 initializes the call with the user114, the computing device 116 can provide pre-call content 102 and/or anotification 106 in anticipation of the user 114 indicating theiracceptance of the call via the computing device 116. The pre-callcontent 102 includes first pre-call content 108 and second pre-callcontent 110, described in more detail below. The notification 106 can bepresented at a display panel 112 of the computing device 116 in responseto the other user 122 initializing the phone call to the user 114.Additionally, or alternatively, the pre-call content 102 can also bepresented to the user 114 and/or the other user 122 via a display panel112 in response to the other user 122 initializing a call to the user114.

The pre-call content 102 can be generated based on data from one or moredifferent sources. In some implementations, the pre-call content 102 canbe generated using one or more trained machine learning models. Forinstance, an input can be applied to a trained machine learning model,and an output that is based on the input being applied to the trainedmachine learning model can be used as a basis for generating thepre-call content 102. The pre-call content 102 can be generated prior tothe other user 122 initializing the call to the user 114, in order toeliminate latency that may otherwise be exhibited. For instance, priorto the other user 122 initializing the call to the user 114, the varioustypes of data can be applied to one or more trained machine learningmodel, such as calendar data and/or message data. Output data, resultingfrom these types of data being applied to the one or more trainedmachine learning models, can be used as a basis for generating firstpre-call content 108 and second pre-call content 110.

For example, in order to generate the first pre-call content 108,calendar data and other data identifying the other user 122 can beapplied to a trained machine learning model. Output data that isprovided as a result can be used as a basis for generating the firstpre-call content 108, which can include calendar data that is associatedwith the other user 122. Furthermore, message data and data identifyingthe other user 122, can be applied to a trained machine learning model.Output data that is provided as a result can be used as a basis forgenerating the second pre-call content 110, which can include messagedata that is also associated with the other user 122. Subsequent togenerating such pre-call content, should the other user 122 initialize acall to the user 114, the pre-call content 102 can be presented to theuser 114 and/or the other user 122 via the computing device 116 and/orthe other computing device 124, respectively.

As an example, and as provided in FIG. 1A, in response to the other user122 initializing the call, the pre-call content 102 can be presented atthe display panel 112. As a result, the user 114 is able to review thepre-call content 102 prior to indicating acceptance of the call. Forinstance, the display panel 112 can be a touch display panel 112 thatallows the user 114 to provide one or more gestures in order to scrollthrough the pre-call content 102 in order to view more of the firstpre-call content 108 and/or view more of the second pre-call content110. In some implementations, the first pre-call content 108 can beinitially displayed and the second pre-call content 110 can be partiallydisplayed, or omitted, until the user 114 provides one or more gesturesindicating the user 114 would be interested in viewing the secondpre-call content 110. For instance, the user 114 can be providing a tapgesture, slide gesture, single touch gesture, multi-touch gesture,spoken gesture, facial gesture, and/or any other gesture that can beprovided as input to a computing device.

When the user 114 indicates acceptance 118 of the incoming call from theother user 122, one or more communication channels can be establishedbetween the computing device 116 and the other computing device 124 viathe network 120. The call can be a voice call, video call, both videoand voice call, and/or any other type of call that can be executed viacomputing devices. As an example, and as provided in FIG. 1A, the callcan be a voice call in which the other user 122 and the user 114 canprovide verbal inputs. For instance, the other user 122 can provide afirst input 126 such as, “Hey, looking forward to the event,” and theuser 114 can provide a second input 128 such as, “Great. Tim wants tomeet beforehand.” The first input 126 and/or the second input 128 can bedialog content of the call between the user 114 and the other user 122.

The dialog content, with prior permission from the user 114 and theother user 122, can be captured as data that can be used as a basis forgenerating subsequent pre-call content 142, as provided in view 130 ofFIG. 1B. For instance, the dialog content can be compared to thepre-call content 102 to determine whether one or more correlationsbetween the dialog content and the pre-call content 102 were apparent. Acorrelation that can be identified may include the second pre-callcontent 110 identifying a message from “Tim” and the user 114 providingthe second input 128 that identifies “Tim.” Additionally, oralternatively, a correlation that can be identified can include theother user 122 providing the first input 126 that identifies “theevent,” and the first pre-call content 108 that identifies the “reunionevent.” In some implementations, one or more correlations can beidentified by comparing data that is based on the dialog content, todata that is based on first pre-call content 108, and also to other datathat is based on the second pre-call content 110. In someimplementations, one or more correlations can be identified by applying,to one or more trained machine learning models, data characterizing thedialog content and other data characterizing the first pre-call content108. Also, one or more correlations can be identified by applying, toone or more trained machine learning models, data characterizing thedialog content and other data characterizing the second pre-call content110.

Other pre-call content 142 can be provided to the user 114 and/or theother user 122 via their respective computing devices in response to theuser 114 and/or the other user 122 initializing a call. For example, asprovided in FIG. 1B, when the computing device 116 is awaitingacceptance of the incoming call from the other user 122, the displaypanel 112 can be caused to provide another notification 144 and otherpre-call content 142. The notification 144 can be an optionalnotification that the user 114 is receiving a request to participate ina call with the other user 122 (e.g., “Charlie”). The other pre-callcontent 142 can be provided at the display panel 112 and can includefirst pre-call content 132 and second pre-call content 134.

The first pre-call content 132 and the second pre-call content 134 canbe provided at the display panel 112 based on whether one or morecorrelations were identified between the dialog content and the firstpre-call content 108, and the dialog content and the second pre-callcontent 110. Furthermore, the arrangement of the first pre-call content132 and the second pre-call content 134 at the display panel 112 can bebased on whether the one or more correlations were identified. Forexample, the “message” type of pre-call content can be assigned a higherpriority than the “calendar” type of pre-call content when a correlationbetween the dialog content and the second pre-call content 110 isidentified, but no correlation between the dialog content and the firstpre-call content 108 is identified. Additionally, or alternatively,“calendar” type pre-call content can be assigned a lower priorityrelative to one or more other types of pre-call content when nocorrelation between the “calendar” type pre-call content and the dialogcontent is identified. As a result, a priority of “calendar” typepre-call content can have a lower priority relative to “social media”type pre-call content, as illustrated in FIG. 1B. As a result, thedisplay panel 112 can present “message” type pre-call content and“social media” type pre-call content when the computing device 116 isawaiting acceptance 136 of the subsequent incoming call from the otheruser 122.

For example, when the computing device 116 provides the notification 144of the incoming call, the user 114 can be presented with the firstpre-call content 108, which can provide content corresponding to variousactivities of one or more messaging applications. For instance, contentfrom two different messaging applications can be consolidated to providethe first pre-call content 132, which can include a portion of a latestmessage from “Charlie,” the other user 122 who initialized the call, and“Tim,” someone that the user 114 mentioned in the dialog content. Theuser 114 can influence subsequent presentations of pre-call content byengaging with the other pre-call content 142 (e.g., thereby causing oneor more trained machine learning models to reflect the positiveengagement by the user), and/or by not engaging with the other pre-callcontent 142 (e.g., thereby causing one or more trained machine learningmodels to reflect the negative engagement by the user). Additionally, oralternatively, positive engagement can be indicated by the user and/orthe other user 122, providing one or more gestures before, during,and/or after the other pre-call content 142 is be presented to the user114.

For example, after acceptance 136 of the subsequent call, the other user122 can provide a first input 138 such as “Hey, what's been going on?”to which the user 114 can respond with a second input 140, “Tim iswanting to get together. Did you see Carol's photos?” Dialog contentcapturing the first input 138 and other dialog content capturing thesecond input 140 can be compared to the other pre-call content 142 todetermine whether one or more correlations are apparent. Because theuser 114 mentioned “Carol's photos” and a message from “Tim,” each ofthe first pre-call content 132 and the second pre-call content 134 canhave positive correlations to the dialog content. As a result, thecontent, type of content, and/or other data related to the otherpre-call content 142 can be treated as encouraging positive engagement,thereby presumably assisting the user 114 before accepting a call. Thiscan allow for more efficient calls between users, as users would nolonger need to access separate applications to be reminded of certaininformation (e.g., recent calendar events and/or messages). This canalso help to ensure that the information is available at a time of need,particularly where communication signals are intermittent. Additionally,because the information may be retrieved from one or more locations insecondary storage (for example, flash memory) and loaded into primarymemory (e.g., volatile RAM) in advance of the user requesting thatinformation, retrieval of that information occurs more quickly fromperspective of the user. This can also result in a decrease in usage ofcomputational resources that might otherwise be consumed while the useris on the phone, and attempting to remember the information, therebyprolonging the call longer than necessary. When pre-call content isprovided before video calls, computational resources can be even morepreserved, relative to audio calls, as many video calls require theexchange of high definition video over the period of the video call.

FIG. 2A, FIG. 2B, and FIG. 2C illustrate a user changing contexts andcausing pre-call content to be generated according to the changes incontext. For example, FIG. 2A illustrates a view 200 of a user 214riding in a vehicle 212 with another user 210. While riding in thevehicle, the user 214 can be engaging in one or more activities with acomputing device 216, which can include a location sensor from which alocation of the computing device 216 can be determined. Additionally,the computing device 216 can include, or otherwise provide access to, anautomated assistant 218. The user 214 can engage with the automatedassistant 218 via one or more gestures and/or one or more spokenutterances. For example, while riding in the vehicle 212, the user 214can provide a spoken utterance 224 such as, “Assistant, tell me therecipe for pecan pie.” In response, the automated assistant 218 canaccess client data 222 and/or one or more applications 220 in order tofulfill the request from the user 214. In order to fulfill the request,the automated assistant 218 can access a browser application of theapplications 220 and download recipe data, which can be stored as partof the client data 222.

The computing device 216 can be in communication with a remote computingdevice 202, such as computing device 204, which can be a server device.The computing device 204, with prior permission from the user 214, cangenerate and/or otherwise access contextual data 206, which cancharacterize activities of the user 214 over time. The contextual data206 characterizing the activity of the other user 214 can be used togenerate pre-call content 208. The pre-call content 208 can be generatedat any time, regardless of whether there is an incoming call coming fromanother user or not. For example, in response to the user 214 invokingthe automated assistant 218 to download the recipe, pre-call content 208can be generated in order to characterize the interaction with theautomated assistant 218, should the user 214 receive an incoming callthat is associated with the interaction. For example, the user 214 mayhave a Derby event to attend on their calendar, and the Derby event maydescribe “pecan pie” as being served at the Derby event. Therefore, thepre-call content 208 can be provided to the user 214 in response to aDerby event invitee (e.g., “Virginia”) calling the user 214. In otherwords, when the computing device 216 is waiting for the user 214 toaccept the call, pre-call content 208 can be provided at a display panelat the computing device 216, in order to refresh a recollection of theuser 214, especially with respect to the pecan pie recipe and/or theDerby event. However, should the user 214 perform other activities thatare temporarily separate from the activities in the vehicle 212, thosesubsequent activities can be used as a basis for generating otherpre-call content 208.

For example, FIG. 2B illustrates a view 230 of the user 214 engaging inactivities within their kitchen 236, subsequent to the user 214 ridingin the vehicle 212. The user 214 can be engaged in activities such asbaking a pecan pie using a first connected device 232, washing dishesvia a second connected device 242, and playing a podcast 234 via aclient device 244, such as a standalone speaker device 240. The user 214can engage in each activity using one or more computing devices, and/oran automated assistant to interact with various devices.

The subsequent activities engaged in by the user 214 can also becharacterized by contextual data 206, and used as a basis from which togenerate additional pre-call content 208. For example, pre-call content208 generated based on the activities depicted in FIG. 2B can include“operational” type data. In this way, should the user 214 receive anincoming call from the Derby event invitee (e.g., “Virginia”), theclient device 244 can audibly render pre-call content 208 in order torefresh the recollection of the user 214 in furtherance of the userhaving a more efficient call related to the Derby event. For example,the client device 244 can render audio characterizing an operationalstatus of the first connected device 232, an operational status of thesecond connected device 242, and a current operating status of theclient device 244, while also providing an audible sound indicating thatan incoming phone call is awaiting acceptance by the user 214. As anexample, the client device 244 can render a ringing sound to indicate anincoming call is being received, while also rendering audio data suchas, “The oven timer has 35 minutes left,” since the user 214 previouslyrequested a recipe that was related to the Derby event, and a Derbyevent invitee is calling the user 213. However, if no call is receivedwhile the user is engaging in the activities, pre-call content 208 canbe otherwise stored and available to be presented at another time in thefuture. It will be appreciated that the above operations may be executedin response to an incoming call being received at a different clientdevice to the client device 244. For example, the client device 244 mayperform the above described operations in response to an incoming callbeing received at a smartphone device. That is, pre-call content may beprovided on a device other than the device at which the incoming call isreceived.

FIG. 2C depicts the user in a context, subsequent to those characterizedin FIG. 2A and FIG. 2B. However, pre-call content generated based on theactivities of the user 214 within each context can be stored andpresented within the context depicted in FIG. 2C. For example, asprovided in view 250 of FIG. 2C, first pre-call content can be generatedbased on data that resulted from the user 214 engaging in the activitydepicted in FIG. 2B, in which the user 214 initialized their oven timer238. The timer could have been initialized the user 214 via a request toa client automated assistant 246 via an assistant interface 248 of theclient device 244. Therefore, when generating first pre-call content262, automated assistant data and/or device data can be accessed inorder to provide relevant pre-call content such as, “Oven timer has 30minutes left.” In some implementations, an incoming call can affect acurrent operating status of a device, such as the client device 244,which can pause an audio output (e.g., podcast audio) in response to anincoming call from another user 254. Therefore, any pre-call contentthat is generated based on operational data characterizing the clientdevice 244 can be updated in response to the incoming call, andreflected at a display panel 266 of a computing device 272. Forinstance, as provided in view 250, pre-call content 260 provided to theuser 214, while the computing device 272 is awaiting acceptance of thecall, can include first pre-call content 262 including “History podcastis paused.”

Furthermore, in response to the other user 254 (e.g., a Derby eventinvitee) initializing a call to the user 214 via computing device 256,second pre-call content 264 can be provided at the display panel 266based on activities of the user 214 described with respect to the FIG.2A. In some implementations, a synopsis of activities and/or contentviewed by the user 214 can be presented as part of the pre-call content.For instance, because the user 214 was viewing a “recipe” via a webbrowser application of the computing device 216, content of the recipeweb page can be applied to a machine learning model for generating datafrom which a synopsis of the web page can be generated. Therefore, as aresult, the pre-call content 260 can include second pre-call content 264such as “Pre-Call Browsing Content” that provides a synopsis of a recentweb page (e.g., “pecan pie Recipe”) viewed by the user 214. Furthermore,other browsing content can also be characterized by the second pre-callcontent 264, such as “Directions to Churchill Downs,” if the user 214had recently requested that the automated assistant provide suchdirections.

Each of the first pre-call content 262 and the second pre-call content264 can correspond to different sources and/or types of pre-callcontent. In this way, an amount of engagement with each type and/orsource of content can be determined, in order that one or more trainedmachine learning models can be trained based on such engagement.Furthermore, the user 214 can engage with the pre-call content 260 inorder to determine whether to accept the call from the other user 254 ornot accept the call. Should the user 214 elect to indicate acceptance268 of the call, the pre-call content 260 can enable the user 214 tohave a more effective call, compared to if the user had not seen thepre-call content 260. For example, the other user 254 can be the Derbyevent invitee, and can provide a first input 274 to the call, such as“Hey, how's the pie coming along?” and in response, the user 214 canprovide a second input 276 such as, “Good. Got the recipe earlier, andoven has 30 minutes left. Listening to a history podcast while I wait!”By providing the pre-call content 260, based on the association of thecaller to the user's 214 calendar event, the user 214 is able to be morereadily responsive during the call, thereby reducing the amount of thetime the user 214 will be on the call, as well as the other user 254being on their respective computing device 256.

FIG. 3 illustrates a system 300 for providing pre-call content to one ormore users that are expected to receive an incoming call. The pre-callcontent can be provided in order to eliminate the need for otherapplication data to be accessed during a call in order to eliminatewaste of energy and/or computational resources during a call. In someimplementations, the pre-call content can be provided via an automatedassistant 308 that is accessible to a client device, and/or via anyother application or module that is access to the client device. Theautomated assistant 308 can operate as part of an automated assistantapplication that is provided at one or more computing devices 302. Auser can interact with the automated assistant 308 via one or moreassistant interfaces 304, which can include one or more of a microphone,a camera, a touch screen display, a user interface, and/or any otherapparatus capable of providing an interface between a user and anapplication. For instance, a user can initialize the automated assistant308 by providing a verbal, textual, and/or a graphical input to theassistant interface to cause the automated assistant 308 to perform afunction (e.g., provide data, control a device (e.g., control an IoTdevice, access an agent, modify a setting, control an application(s),etc.). The computing device 302 can include a display device, which canbe a display panel that includes a touch interface for receiving touchinputs and/or gestures for allowing a user to control applications ofthe computing device 302 via the touch interface. The touch inputsand/or other gestures (e.g., spoken utterances) can also allow a user tointeract with the automated assistant 308.

In some implementations, the computing device 302 can lack a displaydevice but include an audio interface (e.g., a speaker and/or amicrophone), thereby providing an audible user interface output, withoutproviding a graphical user interface output, as well as providing a userinterface, such as a microphone, for receiving spoken natural languageinputs from a user. For instance, in some implementations, the computingdevice can include one or more tactile input interfaces, such as one ormore buttons, and omit a display panel that would be provided graphicaldata from a graphics processing unit (GPU). In this way, energy andprocessing resources can be saved compared to a computing device thatincludes a display panel and GPU.

The computing device 302 can be in communication with a server deviceover a network, such as the internet. The computing device 302 canoffload computational tasks to the server device in order to preservecomputational resources at the computing device 302. For instance, theserver device can host an automated assistant, and the computing device302 can transmit inputs received at one or more assistant interfaces,and/or a user interface, to the server device. However, in someimplementations, the automated assistant 308 can be hosted at thecomputing device 302. In various implementations, all or less than allaspects of the automated assistant 308 can be implemented on the serverdevice and/or the computing device 302. In some of thoseimplementations, aspects of the automated assistant 308 are implementedvia a local automated assistant and interface with the server device,which can implement other aspects of the automated assistant 308. Theserver device can optionally serve a plurality of users and theirassociated assistant applications via multiple threads. Inimplementations where all or less than all aspects of the automatedassistant 308 are implemented via a local automated assistant of thecomputing device 302, the local automated assistant can be anapplication that is separate from an operating system of the computingdevice 302 (e.g., installed “on top” of the operating system)—or canalternatively be implemented directly by the operating system of thecomputing device 302 (e.g., considered an application of, but integralwith, the operating system).

In some implementations, the automated assistant 308 can include aninput processing engine 312, which can employ multiple different enginesfor processing inputs and/or outputs for computing device 302. Forinstance, the input processing engine 312 can include a speechprocessing engine 314 that can process audio data received at anassistant interface 336 to identify the text embodied in the audio data.The audio data can be transmitted from, for example, the computingdevice 302 to the server device in order to preserve computationalresources at the computing device 302.

The process for converting the audio data to text can include a speechrecognition algorithm, which can employ neural networks and/orstatistical models for identifying groups of audio data corresponding towords or phrases. The text converted from the audio data can be parsedby a data parsing engine 316 and made available to the automatedassistant 308 as textual data that can be used to generate and/oridentify command phrases from the user. In some implementations, outputdata provided by the data parsing engine 316 can be provided to anaction engine 318 to determine whether the user has provided an inputthat corresponds to a particular action and/or routine capable of beingperformed by the automated assistant 308 and/or an application, agent,and/or device that is capable of being accessed by the automatedassistant 308. For example, assistant data 322 can be stored at theserver device and/or the computing device 302, and can include data thatdefines one or more actions capable of being performed by the automatedassistant 308, as well as parameters involved in performing thoseactions.

When the input processing engine 312 has determined that a user hasrequested a particular action or routine be performed, the action engine318 can determine one or more parameter values for the particular actionor routine, and an output generating engine 320 can then provide anoutput to the user based on the particular action, routine, and/or theone or more parameters. For instance, in some implementations, inresponse to a user input, such as a gesture directed at the assistantinterface 336, the automated assistant 338 can cause data, whichcharacterizes the gesture, to be transmitted to the server device fordetermining the action that the user is intending the automatedassistant 308 to perform.

The system 300 can further include a pre-call content engine 336 thatgenerates pre-call content for one or more users in anticipation of theone or more users receiving a call from one or more other users. Thepre-call content engine 336 can access client data 306, assistant data322, and/or application data 332 at the computing device 302 and/or anyother device for generating the pre-call content. For instance, thecomputing device 302 can include various applications with which a usercan engage in a variety of different activities, such as, but notlimited to, sending a message to another user and sharing media (e.g.,photos and/or videos) with another user. When a user performs aparticular activity, data associated with that activity can beprocessed, with permission from the user, by a contextual engine 326.The contextual engine 326 can process the data in order to identifycorrelations between the data and one or more other users, which theuser can receive an incoming call from. When the data, which isgenerated based on a context or particular activity, is identified asbeing associated with another user, the contextual engine 326 caninteract with the pre-call content engine 336 in order to cause thepre-call content engine 336 to generate pre-call content based on theidentified data. For example, if the activity includes the user sendinga message and a video to another user, the message and video can be usedas a basis from which to generate pre-call content.

In some implementations, multiple different instances of pre-callcontent can be generated, and each instance of pre-call content can beassigned a priority by a priority engine 332. The assigned priority fora particular instance of pre-call content can indicate a predictedpropriety of the pre-call content to a call that may occur between auser and another user. In other words, the assigned priority for theparticular instance can indicate a prediction of how appropriate thepre-call content is to the call. For instance, when an instance ofpre-call content is generated by the priority engine 332, based on amost recent message between a user and another user, the instance ofpre-call content can be assigned a highest priority relative to otherinstances of pre-call content that correspond to the user and the otheruser. Such assignment of the highest priority can be based at least inpart on the most recent message being the most recent in time messagebetween the user and the other user. In this way, should the other usercall the user, the highest priority pre-call content can be presented tothe user while the call is pending acceptance by the user. Should theuser subsequently engage in another activity associated with the otheruser, and that other activity be different than sending a message (e.g.,creating a calendar event that identifies the user and the other user),additional pre-call content can be generated by the pre-call contentengine 336 based on the other activity.

The additional pre-call content can be assigned a priority by thepriority engine 332 based on historical determinations of the proprietyof the other activity to calls between the user and the other user. Forinstance, dialog content data and pre-call content data can be appliedto one or more trained machine learning model 328 (e.g., trained basedon the historical determinations) in order to determine a measure thatindicates predicted propriety of the pre-call content to the dialogcontent. The measure can be used as a basis for adjusting an assignedpriority of any other instances of pre-call content that correspond tothe same activity as an activity corresponding to the pre-call contentdata. Furthermore, multiple instances of pre-call content data can bestored on the computing device 302, and one or more instances of thepre-call content data can correspond to different other users that maycall the user, or have called the user in the past. For instance,pre-call content data can be generated in response to a particular usercalling the user for the first time, and in expectation that theparticular user will call the user again.

In some implementations, the contextual engine 326 can determine acurrent context of a user and cause the computing device 302 to renderpre-call content according to the current context. For instance, whenthe client data 306 indicates that the user is controlling the computingdevice 302 or another computing device from a headset, the contextualengine 326 can communicate with the pre-call content engine 336 in orderto convert any suitable pre-call content to audio format. In this way,should the user receive an incoming call, the pre-call content can berendered audibly while the computing device 302 is awaiting acceptanceof the call by the user. In some implementations, the client data 306can indicate that the user is driving a vehicle, and therefore may bedifficult to look at a screen. Therefore, based on this indication, thecontextual engine 326 can communicate with the pre-call content engine336 in order to convert any suitable pre-call content to audio format,should the user receive a call while operating the vehicle.

In some implementations, an interaction engine 330 of the system 300 cangenerate data that characterizes an extent to which a user hasinteracted with pre-call content, and that data can be shared with thepriority engine 332. The priority engine 332 can then re-assignpriorities of particular instances of pre-call content according towhether the user engaged with certain pre-call content or not. Forinstance, a user that receives a call from their best friend or spousemay never rely on pre-call content, and therefore never be perceived tointeract with the pre-call content. As a result, the interaction engine330 can share certain data with the priority engine 332 indicating thatpre-call content for the best friend and/or the spouse can have a lowpriority or otherwise be removed. In some implementations, when pre-callcontent generated for particular anticipated callers is assigned a lowerpriority relative to other anticipated callers, and/or for a thresholdperiod of time, pre-call content for those particular anticipatedcallers can be bypassed. In this way, the user will not be shownpre-call content for those anticipated callers from which the user has ahistory of not relying on such pre-call content.

In some implementations, the contextual engine 326 can determine acontext of the user in order to promote privacy of the user whenproviding pre-call content. For instance, the contextual engine 326 candetermine, with prior permission from the user, that the user is awayfrom their home and/or in a public location. Furthermore, theinteraction engine 330 can determine, based on information from thecontextual engine 326, whether the user typically engages with pre-callcontent when away from their home and/or in public places. When the userhas a history of not engaging with pre-call content while away fromtheir home and/or in public places, regardless of the caller, theinteraction engine 330 and/or contextual engine 326 can communicate thispreference of the user to the priority engine 332. In response, thepriority engine 332 can re-assign priorities of certain pre-call contentwhen the user is determined to be away from their home and/or in apublic place. Additionally, or alternatively, the priority engine 332can assign a tag to certain pre-call content indicating that thepre-call content should not be shown until the tag is withdrawn, orshould not be shown while the user is away from their home and/or in apublic place.

In some implementations, various gestures performed by the user whenpre-call content is being presented can be identified by the interactionengine 330. The gestures can be captured at one or more assistantinterfaces 304, and can include voice input, motion gestures, touchgestures, two-dimensional gestures, three-dimensional gestures, and/orany other gestures that can be identified by a computing device.Furthermore, the interaction engine 330 can use gesture data provided bythe assistant interface 304 to distinguish an amount of engagement auser has for certain pre-call content over other pre-call content. Forinstance, a gesture can cause a display interface of the computingdevice 302 to scroll away from first pre-call content in order to makesecond pre-call content appear for a longer period of time.

FIG. 4 illustrates a method 400 for providing pre-call content to a useraccording to one or more different approaches. The method 400 can beperformed by one or more computing devices, applications, and/or anyother apparatus and/or module capable of generating content for a user.The method 400 can include an operation 402 of determining whether anincoming call is being received by a user from another user. Forexample, the user can have a portable computing device, such as a cellphone, capable of communicating via a wireless communication protocolwith a separate portable computing device associated with the otheruser. When the other user initializes a phone call to the user, one ormore network connections can be established for providing acommunication channel between the portable computing device and theother portable computing device. Data generated in response to the otheruser initializing the phone call can be used as a basis for much todetermine whether an incoming call to the user is being received fromthe other user. When, at operation 402, a determination is made that theuser is receiving a call from another user, the method 400 can proceedto an operation 404.

The operation 404 can include a determination of whether the user hasaccepted the call from the other user. In some implementations, the usercan indicate acceptance of the call from the other user by providing auser input to the portable computing device, and/or any other computingdevice that is associated with the portable computing device. Forexample, the user can indicate acceptance of the call by tapping agraphical element presented at a touch display panel have the portablecomputing device. Alternatively, the user can indicate acceptance of thecall by providing a user input to a computing device that is separatefrom the portable computing device, such as a stand-alone speakerdevice, which can receive spoken utterances for indicating a willingnessto accept phone calls.

When the user has not accepted the phone call, the method 400 canproceed to operation 406 of presenting pre-call content at an interfaceof the computing device (e.g., the portable computing device). Thepre-call content can be generated based on data that is associated withthe user that is receiving the call and/or the other user that hasinitialized the call. Alternatively, or additionally, when the user isthe one who initialized the call, the pre-content can be presented atthe interface of the computing device while the user is waiting for theother user to indicate acceptance of the initialized call. The pre-callcontent can be generated for indicating various different subjectmatter. For instance, the pre-call content can be generated forindicating one or more predicted subjects that will be discussed duringthe phone call. Alternatively, or additionally, the pre-call content canbe generated for attempting to refresh a recollection of the user and/orthe other user prior to the users participating in the phone call.Alternatively, or additionally, the pre-call content can be generated inorder to provide guidance to the user regarding whether to even acceptthe call or not, for instance, based on security concerns, past userexperiences with calls from the other user, a current context of theuser and/or the other user, and/or any other information from which auser can determine whether to accept an incoming call.

The method 400 can proceed from the operation 406 to the operation 408,which can include determining user engagement with the pre-call content.In some implementations, the amount of engagement with pre-call contentcan be based on one or more user inputs provided to one or morecomputing devices that provided and/or are otherwise associated with thepre-call content. In some implementations, the amount of engagement withpre-call content can be determined based on other user activities thatcan be associated with the pre-call content. In some implementations,user engagement with pre-call content can be determined based on anamount of time that the user caused first pre-call content to bepresented at an interface of the computing device, and a separate amountof time that the user caused second pre-call content to be presented atthe interface of the computing device. In the aforementioned example,when the amount of time that the first pre-call content was presented isgreater than the separate amount of time that the second pre-callcontent to be presented, the first pre-call content can be designated asmore relevant to calls between the user and the other user than thesecond pre-call content. In some implementations, relevance of pre-callcontent to certain calls can be provided as positive feedback and/ornegative feedback to a trained machine learning model that can be usedas a basis for creating output from which call content can be based.

The method 400 can proceed from the operation 408 to the operation 410,which can include determining whether the call has been terminated. Forexample, if the user is determined to have not accepted the call atoperation 404, and subsequently the user or the other user haveterminated the call, the method 400 can proceed from the operation 410back to the operation 402. In other words, the user would not haveaccepted the call, and therefore the method 400 can return todetermining whether there is an incoming call. In some implementations,if the user is determined to have engaged with the pre-call contentprior to terminating the call, the pre-call content, and/or a source ofthe pre-call content can be determined as relevant for the user Whenmaking decisions about whether to accept or terminate incoming callsfrom the other user. However, if the user is determined to have notengaged with the pre-call content prior to terminating the call, thepre-call content, and/or the source of the pre-call content, can bedetermined to be irrelevant to the user when the user is makingdecisions about whether to accept or terminate incoming calls from theother user. In some implementations, each pre-call content, type ofpre-call content, and/or source of pre-call content can be assigned apriority, which can be dynamically modified according to whether theuser engages with such pre-call content. Therefore, determinations ofengagement can influence whether the user will be presented with certainpre-call content, types of pre-call content, and/or content from certainsources of pre-call content. For example, if a user engages withpre-call content that is based on a source such as a calendarapplication, that source can be assigned a higher priority than anothersource, such as a GPS source of location data.

When the user has not terminated the call at operation 410, the method400 can continue at operation 404 where the determination is made as towhether the user has accepted the incoming call. When the user isdetermined to have accepted the call at operation 404, the method 400can optionally proceed to the operation 410. This optional path canresult in the user accepting the call, and subsequently awaiting thecall to be discontinued. For example, if the call has been undertakenfor a few minutes, the call can then be terminated at the operation 410,and the method 400 can proceed to the operation 402. When the method 400proceeds in this way, certain pre-call content, types of pre-callcontent, and/or sources of pre-call content can be reprioritizedaccording to user engagement with the pre-call content prior toaccepting the call. In this way, the next time the user receives anincoming call from the other user, the relevancy of the pre-call contentwould be optimized compared to the previous pre-call content presentedto the user.

In some implementations, the method 400 can proceed from the operation404 to the operation 412, after the user is determined to have acceptedthe call at the operation 404. The operation 412 can includedetermining, with user permission, dialog content of the first callbetween the user and the other user. For example, audio datacorresponding to spoken inputs by the user can be captured and stored asdialog content of the call between the user and the other user.Alternatively, or additionally, audio data corresponding to spokeninputs from both the user and the other user can be captured and storedas dialog content of the call between the user and the other user. Insome implementations, the audio data can be transcribed and furtherprocessed as contextual data corresponding to call between the user andthe other user.

The method 400 can proceed from the operation 412 to the operation 414,which can include determining whether any of the dialogue content isrelated to the pre-call content presented to the user and/or the otheruser. For example, the audio data capturing the dialogue content can becompared to the pre-call content to determine whether one or morecorrelations are apparent. In some implementations, textual datatranscribed from the audio data, with permission from the user, can becompared to the pre-call content in order to determine whether one ormore correlations are apparent. As an example, the pre-call content caninclude a synopsis of a calendar event stored by a calendar application,and the calendar event can identify the user and the other user. Whenthe dialogue content includes at least some amount of datacharacterizing at least some portion of the synopsis, the dialog contentcan be considered correlated to the pre-call content.

If one or more correlations are identified between the dialogue contactand the pre-call content, the method 400 can proceed to an operation 406of providing positive feedback to one or more trained machine learningmodels. In this way, one or more machine learning models can be adaptedto provide more relevant content prior to calls between the user and theother user. Furthermore, the method 400 proceed from the operation 416to the operation 420. The operation 422 include generating pre-callcontent based on one or more trained machine learning models. Pre-callcontent can be generated before or after first call has been terminated,but before a second call has been initialized. In this way, latencybetween a call being initialized and the pre-call content beingpresented can be eliminated.

In some implementations, the method 400 can proceed from the operation414 to the operation 418, when one or more correlations between thedialogue content and the pre-call content are not identified. Forexample, if the pre-call content included text from one or morecorrespondence between the user and the other user, and the dialogcontent does not mention any of the text, or anything synonymous withthe text, no correlation would be identified. As a result, the method400 can proceed to the operation 418 of providing negative feedback toone or more trained machine learning models. By providing negativefeedback data to the one or more trained machine learning models, theone or more trained machine learning models can be adapted to providemore relevant data from which pre-call content can be generated. Forinstance, if a source from which pre-call content was generated resultsin no correlation being identified, the one or more trained machinelearning models can be adapted to rely on that source data less often.In some implementations, the one or more trained machine learning modelscan receive as input, identifiers for participants on an incoming call,and available sources for pre-call content. Thereafter, the source thatprovides content that was most relevant can be assigned a higherpriority than another source of content that was not relevant. In someimplementation, priorities can correspond to one or more values foradjusting and/or otherwise training one or more trained machine learningmodels. For example, a node of a machine learning model associated witha particular type of pre-call content data source can have an adjustablepriority value, which can influence a final arrangement of pre-callcontent that is generated for particular calls.

The method 400 can proceed from the operation 418 to the operation 420,in which pre-call content is generated based on the one or more machinelearning models. In some implementations, one or more activities that auser engages in prior to the next incoming call and/or outgoing call canbe used as a basis from which to further train the one or more machinelearning models. Alternatively, or additionally, one or more activitiesthat the user engages in prior to the next incoming call and/or outgoingcall can be used as a basis from which to generate pre-call content tobe presented when the user is being notified of the next incoming call.

FIG. 5 is a block diagram of an example computer system 510. Computersystem 510 typically includes at least one processor 514 whichcommunicates with a number of peripheral devices via bus subsystem 512.These peripheral devices may include a storage subsystem 524, including,for example, a memory 525 and a file storage subsystem 526, userinterface output devices 520, user interface input devices 522, and anetwork interface subsystem 516. The input and output devices allow userinteraction with computer system 510. Network interface subsystem 516provides an interface to outside networks and is coupled tocorresponding interface devices in other computer systems.

User interface input devices 522 may include a keyboard, pointingdevices such as a mouse, trackball, touchpad, or graphics tablet, ascanner, a touchscreen incorporated into the display, audio inputdevices such as voice recognition systems, microphones, and/or othertypes of input devices. In general, use of the term “input device” isintended to include all possible types of devices and ways to inputinformation into computer system 510 or onto a communication network.

User interface output devices 520 may include a display subsystem, aprinter, a fax machine, or non-visual displays such as audio outputdevices. The display subsystem may include a cathode ray tube (CRT), aflat-panel device such as a liquid crystal display (LCD), a projectiondevice, or some other mechanism for creating a visible image. Thedisplay subsystem may also provide non-visual display such as via audiooutput devices. In general, use of the term “output device” is intendedto include all possible types of devices and ways to output informationfrom computer system 510 to the user or to another machine or computersystem.

Storage subsystem 524 stores programming and data constructs thatprovide the functionality of some or all of the modules describedherein. For example, the storage subsystem 524 may include the logic toperform selected aspects of method 400, and/or to implement one or moreof computing device 116, computing device 124, computing device 216,computing device 204, remote computing device 202, client device 244,computing device 272, computing device 256, computing device 302, and/orany other device, operation, application, and/or engine discussedherein.

These software modules are generally executed by processor 514 alone orin combination with other processors. Memory 525 used in the storagesubsystem 524 can include a number of memories including a main randomaccess memory (RAM) 530 for storage of instructions and data duringprogram execution and a read only memory (ROM) 532 in which fixedinstructions are stored. A file storage subsystem 526 can providepersistent storage for program and data files, and may include a harddisk drive, a floppy disk drive along with associated removable media, aCD-ROM drive, an optical drive, or removable media cartridges. Themodules implementing the functionality of certain implementations may bestored by file storage subsystem 526 in the storage subsystem 524, or inother machines accessible by the processor(s) 514.

Bus subsystem 512 provides a mechanism for letting the variouscomponents and subsystems of computer system 510 communicate with eachother as intended. Although bus subsystem 512 is shown schematically asa single bus, alternative implementations of the bus subsystem may usemultiple busses.

Computer system 510 can be of varying types including a workstation,server, computing cluster, blade server, server farm, or any other dataprocessing system or computing device. Due to the ever-changing natureof computers and networks, the description of computer system 510depicted in FIG. 5 is intended only as a specific example for purposesof illustrating some implementations. Many other configurations ofcomputer system 510 are possible having more or fewer components thanthe computer system depicted in FIG. 5.

In situations in which the systems described herein collect personalinformation about users (or as often referred to herein,“participants”), or may make use of personal information, the users maybe provided with an opportunity to control whether programs or featurescollect user information (e.g., information about a user's socialnetwork, social actions or activities, profession, a user's preferences,or a user's current geographic location), or to control whether and/orhow to receive content from the content server that may be more relevantto the user. Also, certain data may be treated in one or more waysbefore it is stored or used, so that personal identifiable informationis removed. For example, a user's identity may be treated so that nopersonal identifiable information can be determined for the user, or auser's geographic location may be generalized where geographic locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular geographic location of a user cannot be determined.Thus, the user may have control over how information is collected aboutthe user and/or used.

In some implementations, a method implemented by one or more processorsis provided and includes determining that a first call between a userand another user has been initialized via a wireless communicationprotocol between a computing device and another computing device. Themethod further includes, when the computing device is waiting for theuser and/or the other user to indicate acceptance of the first call,causing pre-call content to be presented at an interface of thecomputing device. The pre-call content is optionally determined based ondata that is generated using one or more trained machine learningmodels. The method further includes determining, subsequent todetermining that the first call was initialized between the user and theother user, dialog content of the first call. The dialog content isbased on one or more natural language inputs provided by the user and/orthe other user during the first call. The method further includesdetermining, based on the dialog content, whether the pre-call contentincluded at least one correlation to the dialog content. The methodfurther includes determining, subsequent to determining whether thepre-call content included at least one correlation to the dialogcontent, that a second call between the user and the other user has beeninitialized. The method further includes, when the computing deviceand/or the other computing device is waiting for the user and/or theother user to indicate acceptance of the second call: causing theinterface of the computing device, and/or another interface of anothercomputing device, to present or omit the pre-call content based onwhether the pre-call content included the at least one correlation tothe dialog content.

These and other implementations of the technology disclosed herein caninclude one or more of the following features.

In some implementations, the method further includes, when the pre-callcontent is determined to include at least the one correlation to thedialog content: causing the one or more trained machine learning modelsto be further trained according to feedback data indicating a positivecorrelation between dialog content and the pre-call content; and whenthe pre-call content is determined to not include at least onecorrelation to the dialog content: causing the one or more trainedmachine learning models to be further trained according to otherfeedback data indicating a negative correlation between dialog contentand the pre-call content.

In some implementations, the one or more trained machine learning modelsare stored on the computing device.

In some implementations, the method further includes, when the computingdevice and/or the other computing device is waiting for the user and/orthe other user to indicate acceptance of the second call, and thepre-call content is presented at the interface of the computing device,and/or the other interface of the other computing device: causing thepre-call content at the interface of the computing device, and/or theother interface of the other computing device, to be arranged based onwhether the pre-call content included the at least one correlation tothe dialog content.

In some implementations, the pre-call content is generated in advance ofthe first call being initialized. In some versions of thoseimplementations, the pre-call content is generated locally at the clientdevice. In some additional or alternative versions of thoseimplementations, the pre-call content is most recently generatedpre-call content, generated in a most recent iteration of generatingpre-call content relevant to the user and the other user. In some ofthose additional or alternative versions, the most recent iteration isperformed at a time that is based on a measure of frequency and/orquantity of communications between the user and the other user and/orthat is based on new content relevant to the user and/or the other userbeing received. By generating the content in advance of the first callbeing initialized, latency in presenting the first content is reduced,as is latency in accepting the first call. Moreover, when new pre-callcontent is iteratively generated for a given user based on frequency ofcommunication with the given user and/or based on new content fromand/or related to the user being received, new pre-call content isgenerated dynamically in a manner that is tailored to enable pre-callcontent to be fresh, but to conserve resources so that new pre-callcontent is not continuously generated for all users. For example, newpre-call content can be generated at a first higher frequency for agiven user that a user typically communicates with daily, whereaspre-call content can be generated at a second lower frequency for anadditional given user that the user typically communicates with onlyonce a week.

In some implementations, a method implemented by one or more processorsis provided and includes determining content for provisioning at aninterface of a computing device when a user is receiving an incomingcall from another user. The content includes first content data, whichcorresponds to a first assigned priority, and second content data, whichcorresponds to a second assigned priority. The first assigned priorityand the second assigned priority indicate propriety of the content data,at least relative to the incoming call. The method further includesdetermining, subsequent to determining the content for provisioning atthe interface of the computing device and in response to a wirelesstransmission initialized by the other user via another computing device,that the other user has initialized a call to the user via a wirelesscommunication protocol. The method further includes, when the computingdevice is waiting for the user to indicate acceptance of the call:causing, in response to determining that the other user has initializedthe call, the interface of the computing device to present the firstcontent data and the second content data, where the first content datais prioritized over the second content data at least based on the firstassigned priority and the second assigned priority. The method furtherincludes determining, based on the provisioning of the first contentdata, engagement data that characterizes an amount of engagementexhibited by the user when the interface of the computing deviceprovided the first content data, and another amount of engagementexhibited by the user when the interface of the computing deviceprovided the second content data. The method further includesdetermining, based on the engagement data, a third assigned priority forthe first content data and a fourth assigned priority for the secondcontent data. The third assigned priority and the fourth assignedpriority indicate a different propriety of the content data, at leastrelative to a subsequent incoming call from the other user. The methodfurther includes determining, subsequent to determining the thirdassigned priority and the fourth assigned priority, that the other userhas initialized the subsequent incoming call to the user. The methodfurther includes, when the computing device is waiting for the user toindicate acceptance of the subsequent call: causing, in response todetermining that the other user has initialized the subsequent incomingcall to the user, the interface of the computing device to present oromit the first content data and/or the second content data, at leastbased on the third assigned priority and the fourth assigned priority.

These and other implementations of the technology disclosed herein caninclude one or more of the following features.

In some implementations, the engagement data is based on one or moregestures received at the interface of the computing device when theinterface of the computing device is providing the first content dataand the second content data.

In some implementations, the amount of engagement exhibited by the userwhen the interface of the computing device provided the first contentdata is based on a first gesture of the one or more gestures, and theother amount of engagement exhibited by the user when the interface ofthe computing device provided the second content data is based on asecond gesture of the one or more gestures. In some versions of thoseimplementations, the method further includes determining that the firstgesture caused the first content data to be provided at the interface ofthe computing device for a longer period of time than the second gesturecaused the second content data to be provided at the interface of thecomputing device. In those versions, the third assigned priority isprioritized over the fourth assigned priority at least based on thefirst content of data being provided at the interface of the computingdevice for a longer period of time than the second content data.

In some implementations, the method further includes accessing, beforedetermining the content for provisioning to the user, data that at leastcharacterizes one or more features of a context of a user. In some ofthose implementations, the content is determined based on the one ormore features of the context of the user.

In some implementations, determining the third assigned priority for thefirst content data and the fourth assigned priority for the secondcontent data includes: providing, to one or more trained machinelearning models, input data that is based on the first content data andthe second content data; and generating, based on output data generatedby processing the input data using the one or more trained machinelearning models, the third assigned priority and the fourth assignedpriority. In some of those implementations, the method further includescausing the one or more trained machine learning models to be furthertrained based on the engagement data and the other engagement data.

In some implementations, a method implemented by one or more processorsis provided and includes generating first pre-call content forprovisioning at an interface of a computing device when a user isreceiving, via the computing device, an incoming call from another user.The first pre-call content is based on data that resulted from the userengaging in an activity that is associated with the other user. Themethod further includes determining, subsequent to generating the firstcontent data, that the user has engaged in another activity, where theother activity is also associated with the other user and resulted inother data being generated. The method further includes generating,based at least on the other data that resulted from the user engaging inthe activity, second pre-call content in furtherance of provisioningcontent at the interface of the computing device when the user isreceiving, via the computing device, the incoming call from the otheruser. The method further includes determining, subsequent to generatingthe second pre-call content, that the other user has initialized a callto the user via a wireless communication protocol. The method furtherincludes, when the computing device is waiting for the user to indicateacceptance of the call: causing, in response to determining that theother user has initialized the call to the other, the computing deviceto present the first pre-call content and/or the second pre-call contentat the interface of the computing device.

These and other implementations of the technology disclosed herein caninclude one or more of the following features.

In some implementations, generating the first pre-call content includes:providing, based on the data, an input to a trained machine learningmodel, where the first pre-call content includes a synopsis of the dataand the synopsis is generated based on an output generated based onprocessing the input using the trained machine learning model. In someof those implementations, generating the second pre-call contentincludes: processing the other data using the trained machine learningmodel to generate another output based on the other data, where thesecond pre-call content includes another synopsis of the other data andis generated based on the other output of the trained machine learningmodel.

In some implementations, the first pre-call content corresponds to afirst assigned priority and the second pre-call content corresponds to asecond assigned priority, and the activity and the other activity aretemporally distinct activities.

In some implementations, the method further includes, when the computingdevice is waiting for the user to indicate acceptance of the call:determining, based on the first assigned priority and the secondassigned priority, whether to cause the interface of the computingdevice to prioritize a position of the first pre-call content overanother position of the second pre-call content.

In some implementations, the interface of the computing device includesan audio interface and causing the computing device to present the firstpre-call content and/or the second pre-call content at the interface ofthe computing device includes: causing the interface to render audiocorresponding to the first pre-call content and/or the second pre-callcontent when the computing device is waiting for the user to indicateacceptance of the call.

In some implementations, the other user initialized the call via anothercomputing device, and the method further includes, when the computingdevice is waiting for the user to indicate acceptance of the call:determining, in response to determining that the other user hasinitialized the call to the other, whether to cause the interface of thecomputing device to present any pre-call content.

In some implementations, the activity corresponds to the userinteracting with an application and the other activity corresponds tothe user interacting with another application that is different from theapplication.

In some implementations, the method further includes: determining,subsequent to causing the computing device to present the first pre-callcontent and/or the second pre-call content at the interface of thecomputing device, that the user has elected to not accept the call; andcausing, based on determining that the user elected not to accept thecall, third pre-call content to be generated, wherein the third pre-callcontent is presented via the interface of the computing device when thecomputing device is waiting for the user to indicate acceptance of asubsequent call from the other user.

We claim:
 1. A method implemented by one or more processors, the methodcomprising: identifying certain data corresponding to a second user,where the certain data is generated based on further activity by one ormore further users that are distinct from a first user and from thesecond user, and where the further activity by the one or more furtherusers is responsive to the one or more further users receiving anincoming call from the second user; generating, based on the certaindata corresponding to the second user, pre-call content for provisioningat an interface of a computing device when a first user is receiving,via the computing device, an incoming call from a second user;determining, subsequent to the generating the pre-call content, that thesecond user has initialized the call to the first user via a wirelesscommunication protocol; and when the computing device is waiting for thefirst user to indicate acceptance of the call: causing, in response todetermining that the second user has initialized the call to the firstuser, the computing device to present the pre-call content at theinterface of the computing device.
 2. The method of claim 1, wherein thecertain data generated based on the further activity by the one or morefurther users is based on the one or more further users limiting theirengagement with the second user.
 3. The method of claim 1, whereingenerating, based on the certain data corresponding to the second user,the pre-call content includes: providing, based on the certain datacorresponding to the second user, an input to a trained machine learningmodel, wherein the pre-call contentment includes a synopsis of thecertain data and the synopsis is generated based on an output generatedbased on processing the input using the trained machine learning model.4. The method of claim 1, wherein the interface of the computing deviceincludes an audio interface and causing the computing device to presentthe pre-call content at the interface of the computing device includes:causing the interface to render audio corresponding to the pre-callcontent.
 5. The method of claim 1, further comprising: determining,subsequent to causing the computing device to present the pre-callcontent at the interface of the computing device, that the first userhas not accepted the call; and causing additional certain data to begenerated based on determining that the first user elected not to acceptthe call.
 6. A non-transitory computer readable storage mediumconfigured to store instructions that, when executed by one or moreprocessors, cause the one or more processors to perform operations thatinclude: identifying certain data corresponding to a second user, wherethe certain data is generated based on further activity by one or morefurther users that are distinct from a first user and from the seconduser, and where the further activity by the one or more further users isresponsive to the one or more further users receiving an incoming callfrom the second user; generating, based on the certain datacorresponding to the second user, pre-call content for provisioning atan interface of a computing device when a first user is receiving, viathe computing device, an incoming call from a second user; determining,subsequent to the generating the pre-call content, that the second userhas initialized the call to the first user via a wireless communicationprotocol; and when the computing device is waiting for the first user toindicate acceptance of the call: causing, in response to determiningthat the second user has initialized the call to the first user, thecomputing device to present the pre-call content at the interface of thecomputing device.
 7. The non-transitory computer readable storage mediumof claim 6, wherein the certain data generated based on the furtheractivity by the one or more further users is based on the one or morefurther users limiting their engagement with the second user.
 8. Thenon-transitory computer readable storage medium of claim 6, whereingenerating, based on the certain data corresponding to the second user,the pre-call content includes: providing, based on the certain datacorresponding to the second user, an input to a trained machine learningmodel, wherein the pre-call contentment includes a synopsis of thecertain data and the synopsis is generated based on an output generatedbased on processing the input using the trained machine learning model.9. The non-transitory computer readable storage medium of claim 6,wherein the interface of the computing device includes an audiointerface and causing the computing device to present the pre-callcontent at the interface of the computing device includes: causing theinterface to render audio corresponding to the pre-call content.
 10. Thenon-transitory computer readable storage medium of claim 6, wherein theinstructions further include: determining, subsequent to causing thecomputing device to present the pre-call content at the interface of thecomputing device, that the first user has not accepted the call; andcausing additional certain data to be generated based on determiningthat the first user elected not to accept the call.
 11. A systemcomprising: one or more processors; and memory configured to storeinstructions that, when executed by the one or more processors cause theone or more processors to perform operations that include: identifyingcertain data corresponding to a second user, where the certain data isgenerated based on further activity by one or more further users thatare distinct from a first user and from the second user, and where thefurther activity by the one or more further users is responsive to theone or more further users receiving an incoming call from the seconduser; generating, based on the certain data corresponding to the seconduser, pre-call content for provisioning at an interface of a computingdevice when a first user is receiving, via the computing device, anincoming call from a second user; determining, subsequent to thegenerating the pre-call content, that the second user has initializedthe call to the first user via a wireless communication protocol; andwhen the computing device is waiting for the first user to indicateacceptance of the call: causing, in response to determining that thesecond user has initialized the call to the first user, the computingdevice to present the pre-call content at the interface of the computingdevice.
 12. The system of claim 11, wherein the certain data generatedbased on the further activity by the one or more further users is basedon the one or more further users limiting their engagement with thesecond user.
 13. The system of claim 11, wherein generating, based onthe certain data corresponding to the second user, the pre-call contentincludes: providing, based on the certain data corresponding to thesecond user, an input to a trained machine learning model, wherein thepre-call contentment includes a synopsis of the certain data and thesynopsis is generated based on an output generated based on processingthe input using the trained machine learning model.
 14. The system ofclaim 11, wherein the interface of the computing device includes anaudio interface and causing the computing device to present the pre-callcontent at the interface of the computing device includes: causing theinterface to render audio corresponding to the pre-call content.
 15. Thesystem of claim 11, wherein the operations further include: determining,subsequent to causing the computing device to present the pre-callcontent at the interface of the computing device, that the first userhas not accepted the call; and causing additional certain data to begenerated based on determining that the first user elected not to acceptthe call.